Upstream sync 11E: AGC driver onto Metal (29 commits) - #512

…ive COMPUTE_PGM_RSRC1

(cherry picked from commit ee1e341)

(cherry picked from commit 0b5a4aa)

…locks

On a host-imported surface a storage write-back keeps the 64 KiB blocks
the CPU wrote since their tracked layer's generation, stores the rest
and stamps the stored ranges with MarkWritten. Settling advanced the
stored layers' generations past those stamps only when no layer kept a
block (or when an adjacent pending surface needed the collect). When one
tracked layer kept a block, every other stored layer kept its old
generation, so the write-back's own stamps read as CPU writes: the next
write-back kept those layers as well and did not store the GPU's newer
results over them.

A tracked layer is one 64 KiB unit when the surface has block units:
tiled (not linear), 64 KiB-aligned base, not thick, with layer and block
tracking on. Otherwise it is an array layer (64 KiB-aligned layer
slices, layer tracking on) or the whole surface.

Settling now collects and advances each stored layer that kept no
block; a layer that kept a block keeps its old generation and
re-uploads at its next use, as before. When no stored layer advances
(nothing stored, or every selected layer kept a block, such as a single
tracked layer that kept one), no adjacent pending surface needs the collect
and no unit was retiled into the unit shadow, settling still skips the
collect, as main did. Units retiled into the unit shadow are now marked
at the settle collect's generation also when a block was kept; main
marked them at TrackerGeneration() there without collecting. The two
differ only for a CPU write to a stored unit made after the keep
decision: main left it to a later walk, which made the retiled unit
stale so the CPU's bytes won; now the walk stamps it at the marked
generation and the stored results win, as they already did when no
block was kept.

agc_storage_kept_unit_settle uses a 1024x512 RGBA8 surface in 64 KiB
tiling (32 block units): it writes one byte of unit 0 from the CPU,
stores a red clear, stores a blue clear, and requires units 1-31 to
hold blue and unit 0 to keep the CPU's byte after both stores. On main
the GPU-mapped direct memory case fails with unit 1 still red. The
watched-memory case (host imports off: the write-back that stores
through the CPU) passes on main; it guards the kept layer's old
generation and fails when settling also advances a layer that kept a
block. The test returns 77, naming the reason, when the direct-memory
case cannot run: no write watching, no host imports, direct memory not
write-watched, or its host import refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit dc39e05)

…ompared

agc_storage_kept_unit_settle checked that the GPU-mapped direct memory
was write-watched before it made the host import. On Linux, when host
imports resolve write protection and dma-buf imports keep it, the
host-pointer import is tried first in a window that does not stay
watched, and dma-buf only when it fails (importAllocation in
GuestBufferMemory.cpp). Where the host-pointer import of shared memory
succeeds, as on Intel's ANV, the import takes the direct memory out of
the write watch, so the case ran on the compared path: the upload saves
the guest bytes, the first write-back keeps unit 0 by comparison and
drops the saved bytes of every unit it selected, and the second
write-back has nothing to compare. Before e544993 it stored every
unit whole, over the CPU's byte in unit 0, which is the failure
reported on Linux; on main it throws (technical debt: results over
memory outside the write watch without saved bytes). Neither depends
on the settle change. On Windows, importing watched private memory
(compared there) reproduces both: on 7e181fc, with or without this
branch's Texture.cpp change, the second check fails with that message;
on main the second write-back throws.

The case now checks the write watch again after the import and returns
77, naming the reason, when the import took the memory out of it. Where
the import keeps the memory watched (the Windows alias import, a Linux
dma-buf import) it runs as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7bfeeb9)

…ilable

Co-Authored-By: Codex <noreply@openai.com>
(cherry picked from commit ca73d0f)

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 58b2245)

… locks

sceAgcCreateInterpolantMapping resolved the pixel shader's ABI, which takes
the process-wide shader preparation lock until the scope ends, and then
waited under that lock for the vertex shader's background registration
preparation. sceAgcCreatePrimState and sceAgcLinkShaders could wait the same
way. Every other shader registration and ABI resolution queued behind that
one compile: on a shipping title the lock was held 8 to 10 s of every 10 s, almost
all of it waiting for registration preparations, while its loader threads
waited 13 to 45 s per 10 s to take it.

A preparation scope now names the shaders it resolves and waits for their
registration preparations before it can take the lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 11ef8e6)

…location

A storage image whose surface runs past the end of its registered
allocation had every write-back dropped as "memory no longer
registered". Titles can size a single-level surface's allocation as
pitch x height. A tiled mip's 64 KiB blocks are stored row-major, so
that size ends inside the last block row, and the bytes after it are
the tail of that row: they hold texels with y < height, which the GPU's
own store writes into the next allocation. When the allocation ends
inside the last block row and registered readable memory continues to
the surface's end, the write-back now stores the whole surface, as the
GPU does. Padding slots keep their bytes, since the store writes only
bytes that changed.

A shipping title streams its lightmaps into an 8160x8160 BC3 pool
(SW_64KB_S, 2048 block rows) through a 2040x2040 RGBA32UI alias written
by compute. The game allocates 2040 rows, so all its write-backs were
dropped and the walls sampled leftover memory as speckled noise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 6ad9372)

…put groups

The test recompiled and rebuilt the same program for every group of 32
input pairs. Only the input buffer changes between groups, so compile
once per compare, EXEC/VCC capture and float mode (96 programs instead of
768) and dispatch it for each group.

On lavapipe the test drops from about 11.5 s to 1.3 s; it is the slowest
test of the Linux CI run (14.5 s).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 59235d9)

Pointer-shaped calls match the boykopovar#4808 console probe (success, no writes); selecting table 13 for those values was unsupported.

(cherry picked from commit eb1b9ec)

A 16-word compute kernel copies a buffer in 16-byte elements: 64 threads a
group, thread t of group g loads elements g*128+t and g*128+64+t from the
index-addressed buffer in s[0:3] and stores them at the same indices of
s[4:7]. Run as a shader against host imports it costs 11.8 ms of GPU per
present in a shipping title's main menu on an RTX 4060.

Matched like the existing copy kernel, by its code hash and its descriptors
(16-byte stride and no swizzle on both, the same word 3), it becomes one copy
through the existing copy HLE path: the elements the kernel stores, min of
groups * 128 and the destination's record count, when the source's record
count covers them all. APS5_NO_COPY_HLE turns it off with the other kernel.

(cherry picked from commit fc24253)

The stream copy kernel is matched by its 16 code words instead of a hash, and the
match and its range checks become free functions (MatchesStreamCopyKernel,
StreamCopyRange) so a test can reach them. Its loads and stores are format
instructions, so the match also requires the descriptor the pattern fill takes:
32_32_32_32_UINT, XYZW selects, no add-tid, out-of-bounds mode 0, type 0. Copy
paths are counted in copyPathCounts whether or not APS5_PROFILE_DRAW is set.

agc_driver_stream_copy checks the match against every word with a bit flipped,
the descriptor and dispatch refusals, the range clamp and refusals, and runs nine
dispatches through the driver: four must take the HLE path and copy elements
g*128+t and g*128+64+t, five must be refused for their shape and leave the
shader's result.

(cherry picked from commit fa4fc23)

…-image write-back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit db1ddb9)

Build packs from EncodeEntry output and check the hit and miss paths,
an entry stored under another key, a truncated entry, a damaged index,
an out-of-range record, a foreign compiler source version and an
unknown format. A child process covers pack discovery on a real disk
load, the ANYPS5_NO_PREBUILT_SHADERS opt out and fall-through to the
writable cache. ARCHITECTURE.md describes where the prebuilt lookup
sits.

(cherry picked from commit 93ab137)

…rs it reads

Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 5d9def6)

…e dispatch

A captured word that no longer matches a live import or mirror only means the
capture raced a guest write; the GPU reads the live bytes either way, as the
hardware would. The compare now counts such mismatches and reports them with
the snapshot, the region and the first differing offset at most every 10 s
instead of throwing; APS5_SNAPSHOT_CHECK_FATAL=1 restores the throw, and a
snapshot-backed region still fails, now naming the snapshot, the region and
the first differing offset. an Unreal Engine 5 title hit the throw
within 65 to 91 s of reaching its title screen.

(cherry picked from commit 67973cd)

(cherry picked from commit ea13f6a)

(cherry picked from commit 6e3152f)

Apple clang ships libc++, which differs from libstdc++ where the sources
relied on it:

- std::atomic<std::shared_ptr<T>> is not provided. AtomicSharedPtr is that
  type where the library has it and a mutex-guarded pointer with the same
  member functions elsewhere; the AGC driver uses it.
- std::atomic_ref<const T> is not provided; the page span reads go through
  a non-const reference.
- std::filesystem::file_time_type::clock::to_sys is not provided;
  std::chrono::file_clock::to_sys is.
- std::max and std::min do not deduce across the integer types of the
  guest range arithmetic, which is spelled with an explicit type.
- <string> and <initializer_list> are included where the sources and tests
  use them.

(cherry picked from commit facc03a)

…ted memory budget

The storage image cache kept a quarter of the device-local heap (4 GiB on a
16 GiB card) regardless of how much of the heap the driver reports as usable.
With a desktop and other allocations the reported budget is about 10 GiB, the
cache stayed full at 4 GiB, and after minutes of a cutscene device buffer
allocations failed with VK_ERROR_OUT_OF_DEVICE_MEMORY. The cache budget is now
the smaller of that quarter of the heap and a quarter of the reported budget,
at least 512 MiB, read once per second; APS5_TEXTURE_CACHE_MIB still overrides
it. The budget report names the storage budget in use. A test covers the limits.

(cherry picked from commit 64ffeb8)

…ialize

(cherry picked from commit 9960503)

…er and V#

(cherry picked from commit 7af2961)

(cherry picked from commit d16745b)

(cherry picked from commit 843f7e9)

With one sample, PA_SC_AA_MASK leaves one coverage bit per pixel of the 2x2 quad,
so a draw can cover only some pixels of every quad (a shipping title renders a
target one quarter of the pixels at a time this way). The pixel stage info carries that 4-bit
mask; a fragment shader whose mask is not full starts by killing the invocation
unless its gl_FragCoord quad position is covered, and runs without early fragment
tests. The mask is part of the recompile cache key and the serialized request
(fork request version 26). Sample masks of multisampled draws stay refused.

(cherry picked from commit 72518f6)

With PA_SC_AA_MASK covering only some pixels of a quad, the uncovered pixels
run on the hardware as helper lanes, so derivatives of the covered pixels stay
defined. OpKill ends those invocations; OpDemoteToHelperInvocation keeps them
as helpers. The device enables VK_EXT_shader_demote_to_helper_invocation when
available and passes the DemoteToHelperInvocation capability and its SPIR-V
extension to the recompiler target and the graphics shader validation. A
demoted pixel also reads as a helper, so its initial EXEC bit is clear as on the
hardware. Without demote, OpKill stays for pixel programs that read no
neighbouring lanes; implicit-LOD sampling, LOD queries and cross-lane reads are
refused. The quad pixel mask execution test now also samples a mipmapped
texture with implicit LOD under the mask.

(cherry picked from commit 90b20af)

…ask without demote

A program that enables whole-quad mode (s_wqm_b64) and then reads a ballot or
readfirstlane sees the helper lanes, which OpKill removes.

(cherry picked from commit 90783f2)

ReadFirstLane, Ballot and PermuteU32 can consume uncovered helper lanes after a fragment explicitly enables EXEC. Classify those operations as neighbour reads so a partial quad mask refuses their kill fallback when demote is unavailable. ReadFirstLane also stays protected when a constant whole-quad instruction folds away.

The CPU regression covers explicit EXEC, constant WQM, a comparison ballot and ds_permute. Unmasked programs still compile, supported masked programs retain demote and their subgroup operation, and constant colour keeps the kill fallback. The recovered source fails the readfirstlane and ballot checks; the corrected source passes both the focused check and shader-memory suite.

Co-Authored-By: Codex <noreply@openai.com>
(cherry picked from commit dbbe282)

72518f6 reads PA_SC_AA_MASK (0x30e, 0x30f) when it decodes the pixel
stage, for the quad pixel mask. The Metal render-state test built its
registers without them, so the decode threw. The fixture now sets both
masks to cover every sample, as CLEAR_STATE does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj